Parallel Simultaneous Perturbation Optimization

Simultaneous Perturbation Newton Algorithms for Simulation Optimization

We present a new Hessian estimator based on the simultaneous perturbation procedure, that requires three system simulations regardless of the parameter dimension. We then present two Newton-based simulation optimization algorithms that incorporate this Hessian estimator. The two algorithms differ primarily in the manner in which the Hessian estimate is used. Both our algorithms do not compute t...

Simultaneous Perturbation Particle Swarm Optimization and Its FPGA Implementation

The particle swarm optimization technique is one of the promising tools to find a proper optimum for an unknown function optimization. Especially, global search capability of the method is very powerful. The particle swarm optimization utilizes common knowledge of the group and individual experiences effectively. That is, direction for the best estimator that a particle has ever reached, direct...

Convergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization

In this paper, we consider Simultaneous Perturbation Stochastic Approximation (SPSA) for function minimization. The standard assumption for convergence is that the function be three times differentiable, although weaker assumptions have been used for special cases. However, all work that we are aware of at least requires differentiability. In this paper, we relax the differentiability requireme...

An Overview of the Simultaneous Perturbation Method for Efficient Optimization

ultivariate stochastic optimization plays a major role in the analysis and control of many engineering systems. In almost all real-world optimization problems, it is necessary to use a mathematical algorithm that iteratively seeks out the solution because an analytical (closed-form) solution is rarely available. In this spirit, the “simultaneous perturbation stochastic approximation (SPSA)” met...

Perturbation, Optimization and Statistics